FEAT: Adding Garak prompt injection scenario - #2509
FEAT: Adding Garak prompt injection scenario#2509Richard Lundeen (richlundeen) wants to merge 7 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4dcdc508-9524-4451-8291-45176340e841
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4dcdc508-9524-4451-8291-45176340e841
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4dcdc508-9524-4451-8291-45176340e841
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4dcdc508-9524-4451-8291-45176340e841
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4dcdc508-9524-4451-8291-45176340e841
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4dcdc508-9524-4451-8291-45176340e841
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4dcdc508-9524-4451-8291-45176340e841
| for group in context.seed_groups | ||
| if (group.objective.metadata or {}).get("goal_text") == goal_text | ||
| ] | ||
| if not seed_groups: |
There was a problem hiding this comment.
Could we make sure every requested goal is represented in the sample rather than silently skipping goals with no sampled groups?
For example, with:
--goal-texts "goal one" "goal two" --max-dataset-size 1
sampling happens across all context/goal combinations first, so only one goal survives. The other goal reaches this branch and is omitted from the scan even though the user explicitly requested it.
Could we either sample contexts before crossing them with every goal, use stratified sampling, or reject caps smaller than the number of requested goals? A regression test with two goals and a cap of one would make this behavior clear.
| "PromptInject dataset selection only supports prompt_inject_contexts; " | ||
| "technique templates are loaded automatically." | ||
| ) | ||
| return PromptInjectDatasetConfiguration( |
There was a problem hiding this comment.
Could we preserve the caller's full dataset configuration here instead of rebuilding only part of it?
This copies max_dataset_size and filters, but drops settings such as custom validators and auto_fetch=False, as well as behavior from a configuration subclass. For example, a validator supplied by the caller is never run because the new configuration replaces it.
If only PromptInjectDatasetConfiguration is supported, could we reuse that instance or copy all of its relevant configuration? Otherwise, rejecting unsupported configuration types explicitly would be safer than silently changing their behavior.
3 participants
Adds a Garak PromptInject scenario with five selectable injection techniques and three default goal texts. Stores the carrier contexts and injection templates as compact local YAML datasets. Uses standard dataset sampling and persisted objective hashes for reliable scenario resume behavior. Adds focused tests and scanner documentation.